Looking Deeper into Deep Learning Model: Attribution-based Explanations of TextCNN

Layer-wise Relevance Propagation (LRP) and saliency maps have been recently used to explain the predictions of Deep Learning models, specifically in the domain of text classification. Given different attribution-based explanations to highlight relevant words for a predicted class label, experiments based on word deleting perturbation is a common evaluation method. This word removal approach, however, disregards any linguistic dependencies that may exist between words or phrases in a sentence, which could semantically guide a classifier to a particular prediction. In this paper, we present a feature-based evaluation framework for comparing the two attribution methods on customer reviews (public data sets) and Customer Due Diligence (CDD) extracted reports (corporate data set). Instead of removing words based on the relevance score, we investigate perturbations based on embedded features removal from intermediate layers of Convolutional Neural Networks. Our experimental study is carried out on embedded-word, embedded-document, and embedded-ngrams explanations. Using the proposed framework, we provide a visualization tool to assist analysts in reasoning toward the model's final prediction.Comment: NIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services: the Impact of Fairness, Explainability, Accuracy, and Privacy, Montr\'eal, Canad

Looking deeper into deep learning model:attribution-based explanations of TextCNN

Layer-wise Relevance Propagation (LRP) and saliency maps have been recently used to explain the predictions of Deep Learning models, specifically in the domain of text classification. Given different attribution-based explanations to highlight relevant words for a predicted class label, experiments based on word deleting perturbation is a common evaluation method. This word removal approach, however, disregards any linguistic dependencies that may exist between words or phrases in a sentence, which could semantically guide a classifier to a particular prediction. In this paper, we present a feature-based evaluation framework for comparing the two attribution methods on customer reviews (public data sets) and Customer Due Diligence (CDD) extracted reports (corporate data set). Instead of removing words based on the relevance score, we investigate perturbations based on embedded features removal from intermediate layers of Convolutional Neural Networks. Our experimental study is carried out on embedded-word, embedded-document, and embedded-ngrams explanations. Using the proposed framework, we provide a visualization tool to assist analysts in reasoning toward the model's final prediction

Discovery of novel trichomonacidals using LDA-driven QSAR models and bond-based bilinear indices as molecular descriptors

Few years ago, the World Health Organization estimated the number of adults with trichomoniasis at 170 million worldwide, more than the combined numbers for gonorrhea, syphilis, and chlamydia. To combat this sexually transmitted disease, Metronidazole (MTZ) has emerged, since 1959, as a powerful drug for the systematic treatment of infected patients. However, increasing resistance to MTZ, adverse effects associated to high-dose MTZ therapies and very expensive conventional technologies related to the development of new trichomonacidals necessitate novel computational methods that shorten the drug discovery pipeline. Therefore, bond-based bilinear indices, new 2-D bond-based TOMOCOMD-CARDD Molecular Descriptors (MDs), and Linear Discriminant Analysis (LDA) are combined to discover novel antitrichomonal agents. Generated models, using non-stochastic and stochastic indices, are able to classify correctly the 90.11% (93.75%) and the 87.92% (87.50%) of chemicals in the training (test) sets, respectively. In addition, they show large Matthews' correlation coefficients (C) of 0.80 (0.86) and 0.76 (0.71) for the training (test) sets, respectively. The result of predictions on the 10% full-out cross-validation test also evidences the quality of both models. In order to test the models' predictive power, 12 compounds, already proved against Trichomonas vaginalis (Tv), are screened in a simulated virtual screening experiment. As a result, they correctly classified 9 out of 12 (75.00%) and 10 out of 12 (83.33%) of the chemicals, respectively, which were the most important criteria to validate the models. Finally, in order to prove the reach of TOMOCOMD-CARDD approach and to discover new trichomonacidals, these classification functions were applied to a set of eight chemicals which, in turn, were synthesized and tested toward in vitro activity against Tv. As a result, experimental observations confirm theoretical predictions to a great extent, since it is gained a correct classification of 87.50% (7/8) of chemicals. Biological tests also show several candidates as antitrichomonals, since almost all the compounds [VAM2-(3-8)] exhibit pronounced cytocidal activities of 100% at the concentration of 100 mg/mL and at 24 h (48 h) but VAM2-2: 99.37% (100%), and it is remarkable that these compounds do not show toxic activity in macrophage assays at this concentration. The Quantitative Structure-Activity Relationship (QSAR) models presented here could significantly reduce the number of synthesized and tested compounds as well as could act as virtual shortcuts to new chemical entities with trichomonacidal activity. © 2009 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim.Peer Reviewe